1. Field
Certain embodiments of the present disclosure generally relate to neural system engineering and, more particularly, to a method for implementing a digital neural processor with discrete-level synapses and probabilistic synapse weight training.
2. Background
Synapses that connect neurons of a neural system typically exhibit only a small number of discrete strength levels. For example, synapses in the Cornu Ammonis CA3-CA1 pathway of the hippocampus neural structure can be either binary or ternary. In the case of ternary synapse, one strength level can, for example, correspond to a depressed state induced by a long-term depression (LTD), another strength level can correspond to a potentiated state induced by a long-term potentiation (LTP), and a third strength level can correspond to an un-stimulated state that exists prior to applying the LTP or LTD. Transitions between discrete levels of the synapse strength can occur in a switch-like manner.
Neuromorphic processors can be implemented in analog, digital or mixed signal domain. The synapses are typically trained using a deterministic spike-timing-dependent plasticity (STDP) rule, which requires either analog or multi-level digital memory for storing synaptic weights. Either of these two types of memories presents technical challenges. The analog memory has a poor retention and cloning capability. It also requires a large die area for implementing STDP decays. On the other hand, the multi-level digital memory requires a large number of bits, which leads to a large number of parallel interconnect wires for implementing synaptic weight increments of a given accuracy.